Refactor/egress mitmproxy yaml config by Pangjiping · Pull Request #975 · opensandbox-group/OpenSandbox

Pangjiping · 2026-06-04T02:30:36Z

Summary

Move static mitmproxy options out of launch.go hardcodes into a baked-in config.yaml under the standard mitm confdir layout. launch.go keeps only per-deployment dynamic flags
(env-driven --set).

config.yaml lists only deviations from mitm built-in defaults:

mode:
  - transparent
listen_host: 127.0.0.1
stream_large_bodies: 10m
ssl_verify_upstream_trusted_confdir: /etc/ssl/certs
ignore_hosts: []   # default-matching, kept as discoverable extension point

Precedence: --set (env override) > config.yaml > mitm defaults.

Why

Two latent bugs in the launch.go surface:

stream_large_bodies set twice. launch.go wrote 1m, custom.py overrode to 10m via ctx.options. launch.go line was dead.
ignore_hosts multi-value silently overwritten. Each ;-separated entry was a separate --set ignore_hosts=..., and mitm --set on a list option REPLACES the list — only the
last value survived.

config.yaml natively expresses lists, eliminates the double-set, and gives operators a single reviewable file for fleet-wide static defaults.

Removed env vars

OPENSANDBOX_EGRESS_MITMPROXY_CONFDIR — no internal use; would have broken config.yaml discovery.
OPENSANDBOX_EGRESS_MITMPROXY_IGNORE_HOSTS — replaced by yaml-native list (the env path was the source of the silent-overwrite bug).

Override paths (documented)

Downstream image COPY over the baked-in path (recommended).
K8s ConfigMap subPath mount at runtime.
Single-option env --set for the documented dynamic env vars.

Backward compat

/egress symlink → /opt/opensandbox-egress/egress for tooling that still references the pre-#951 path.

Testing

Not run (explain why)
Unit tests
Integration tests
e2e / manual verification

Breaking Changes

None
Yes (describe impact and migration path)

Checklist

Linked Issue or clearly described motivation
Added/updated docs (if needed)
Added/updated tests (if needed)
Security impact considered
Backward compatibility considered

…ynamic) Move fleet-wide, rarely-changing mitmproxy options into a baked-in config.yaml under the standard mitm confdir layout, so launch.go only emits per-deployment dynamic overrides via --set. This eliminates two classes of bug along the way: - stream_large_bodies was set in two places (launch.go --set 1m and custom.py ctx.options 10m), with the addon silently winning — making the launch.go line dead code. Now declared once in config.yaml (10m). - ignore_hosts was env-driven with `;`-separated values, but each value was passed as a separate --set, and mitmproxy --set on a list option REPLACES the list — so configuring multiple bypass patterns silently only kept the last one. config.yaml uses a native YAML list with no override semantics. Static options now in /var/lib/mitmproxy/.mitmproxy/config.yaml: mode, listen_host, connection_strategy (lazy — historical default preserved here; switching to eager is tracked in a separate change), stream_large_bodies (10m), http2, ignore_hosts (empty default), ssl_verify_upstream_trusted_confdir (default). Dynamic overrides remain env-driven and applied as --set in launch.go (precedence: --set > config.yaml > mitm defaults): OPENSANDBOX_EGRESS_MITMPROXY_TRANSPARENT (toggle) OPENSANDBOX_EGRESS_MITMPROXY_PORT OPENSANDBOX_EGRESS_MITMPROXY_SCRIPT OPENSANDBOX_EGRESS_MITMPROXY_SSL_INSECURE OPENSANDBOX_EGRESS_MITMPROXY_UPSTREAM_TRUST_DIR Removed env vars (no internal use, replaced by config.yaml): OPENSANDBOX_EGRESS_MITMPROXY_CONFDIR — confdir is the mitm user's home (/var/lib/mitmproxy), which is also where config.yaml lives; splitting them via env created an unused escape hatch that would have broken config.yaml discovery. OPENSANDBOX_EGRESS_MITMPROXY_IGNORE_HOSTS — replaced by ignore_hosts in config.yaml (native list, no covert-overwrite bug). The mitmproxy.Config struct loses its ConfDir field accordingly. SyncRootCA still accepts an optional confDirEnv argument so the existing candidate-path search behavior is preserved if a future caller needs to plumb it back in.

…ConfigMap mount) The previous draft told operators to edit components/egress/mitmproxy/config.yaml and rebuild — true for the in-repo flow, but does not help operators consuming a published egress image who want different static defaults. Add a section spelling out the three supported override paths: 1. Build a downstream image that COPYs an alternate config.yaml over the baked-in path (recommended: version-controlled, reproducible). 2. Mount an override at /var/lib/mitmproxy/.mitmproxy/config.yaml at runtime (Kubernetes ConfigMap subPath mount example included). 3. Use the env-driven --set escape hatch for the small set of options exposed via environment variables. Also warn against in-container edits, which are lost on restart and blocked by the mitmproxy user's read-only access.

…ibility PR opensandbox-group#951 moved the egress binary from /egress to /opt/opensandbox-egress/egress so the supervisor and binary could share a single grouped directory. External tooling and older deployment manifests may still reference the old /egress path; add a symlink so both paths resolve to the same binary. Symlink rather than COPY: zero extra image size, single source of truth for chmod and replacement, and `exec /egress` resolves to the supervisor-managed binary like before.

Pangjiping

Clean refactor — fixes real bugs (double-set stream_large_bodies, ignore_hosts overwrite). One concern: connection_strategy may have silently changed from lazy to eager since mitmproxy 10 changed the default.

Keep HEAD version of env var table — IGNORE_HOSTS and CONFDIR moved to config.yaml static config, no longer env vars. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

mitmproxy 10+ changed the default from lazy to eager. Pin lazy explicitly to preserve the historical behavior of deferring upstream connections. Also fix comments in launch.go, configuration.go, and the doc that incorrectly claimed connection_strategy matches the mitm default. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

mitmproxy's built-in ignore_hosts in transparent mode matches against the destination IP:port before the TLS handshake — SNI hostname is not yet available at that point. This makes domain-based TLS pass-through unreliable. Add a tls_clienthello hook to the system addon that re-checks the same ignore_hosts patterns against the SNI hostname from the ClientHello. When a match is found, data.ignore_connection=True causes mitmproxy to forward the encrypted connection without interception. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…nfdir When a volume mount at /var/lib/mitmproxy shadows the baked-in config.yaml, operators can point mitmproxy's confdir to an alternate location via env var. mitmdump reads config.yaml and stores CA certs under this directory. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…rride confdir" This reverts commit 5131006.

hittyt · 2026-06-09T09:23:56Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 641542255b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Preserve previous --set stream_large_bodies=1m threshold. 10m would buffer up to 10x more per flow for non-SSE/non-chunked responses in transparent mode, increasing RSS and delaying forwarding under high-concurrency. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

hittyt

LGTM

Pangjiping force-pushed the refactor/egress-mitmproxy-yaml-config branch from 022c537 to 92581d3 Compare June 4, 2026 04:05

Pangjiping added 3 commits June 4, 2026 14:23

Pangjiping force-pushed the refactor/egress-mitmproxy-yaml-config branch from 6b6824c to a90b629 Compare June 4, 2026 07:43

Pangjiping marked this pull request as ready for review June 4, 2026 08:10

Pangjiping requested review from hittyt and jwx0925 as code owners June 4, 2026 08:10

Pangjiping assigned hittyt Jun 4, 2026

Pangjiping added the component/egress label Jun 4, 2026

Pangjiping commented Jun 6, 2026

View reviewed changes

Comment thread components/egress/mitmproxy/config.yaml Outdated

Comment thread components/egress/pkg/mitmproxy/launch.go Outdated

Comment thread components/egress/pkg/constants/configuration.go Outdated

Pangjiping and others added 5 commits June 9, 2026 12:56

docs(egress): resolve merge conflict in mitmproxy-transparent.md

0071c94

Keep HEAD version of env var table — IGNORE_HOSTS and CONFDIR moved to config.yaml static config, no longer env vars. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Revert "feat(egress): add OPENSANDBOX_EGRESS_MITMPROXY_CONFDIR to ove…

6415422

…rride confdir" This reverts commit 5131006.

Pangjiping added the feature New feature or request label Jun 9, 2026

chatgpt-codex-connector Bot reviewed Jun 9, 2026

View reviewed changes

Comment thread components/egress/pkg/mitmproxy/launch.go

Comment thread components/egress/mitmproxy/config.yaml Outdated

Comment thread components/egress/pkg/mitmproxy/launch.go

Comment thread components/egress/pkg/mitmproxy/launch.go

hittyt approved these changes Jun 9, 2026

View reviewed changes

hittyt merged commit f15383c into opensandbox-group:main Jun 9, 2026
10 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor/egress mitmproxy yaml config#975

Refactor/egress mitmproxy yaml config#975
hittyt merged 9 commits into
opensandbox-group:mainfrom
Pangjiping:refactor/egress-mitmproxy-yaml-config

Pangjiping commented Jun 4, 2026 •

edited

Loading

Uh oh!

Pangjiping left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hittyt commented Jun 9, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hittyt left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Pangjiping commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Removed env vars

Override paths (documented)

Backward compat

Testing

Breaking Changes

Checklist

Uh oh!

Pangjiping left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hittyt commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hittyt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Pangjiping commented Jun 4, 2026 •

edited

Loading

hittyt commented Jun 9, 2026 •

edited

Loading